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ABSTRACT 

An intent of many business programs is to enhance the critical thinking capabilities of their 
students. Since AACSB accreditation requires evidence that business schools fulfill their goals, 
our students were required to take the California Critical Thinking Skills Test (CCTST). As 
expected, the CCTST was significantly related to SAT performance and to Business Core GPA. 
Nonetheless, the especially strong relationships with components thought to emphasize critical 
thinking, such as performance in the business capstone course, were not observed. SAT scores 
alone predicted 56% of the variance in CCTST. None of our other predictors added significantly 
to this total. 
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INTRODUCTION 


yy' ncreasingly, the agencies responsible for accrediting programs in higher education have been requiring 
yC direct evidence that universities are fulfilling their stated missions. This is true both for the groups that 
oversee general education, for example, the North Central Association (1991, 2003) and those who 
accredit specific programs such as the AACSB International—The Association to Advance Collegiate Schools of 
Business, (2006). 


As the body that deals exclusively with the accreditation of business programs per se, AACSB publishes 80 
pages of standards (AACSB-International, 2006) to provide schools with detailed guidelines concerning the criteria 
the review team will use in its evaluation. Among other things, certain content areas are expected to be covered in 
both undergraduate and MBA programs. These include ethical and legal responsibilities, financial theories, 
domestic and global economic environments, as well as group and individual organizational dynamics (AACSB- 
International, 2006, p. 71). 

Besides specifying areas of content coverage, AACSB-International (2006) expects schools to provide 
evidence that “learning occurs for each of the learning goals” (AACSB-International, 2006, p. 62). A variety of 
methods are delineated for providing evidence of learning achievement and examples of how these approaches might 
be used (cf., AACSB-International, 2006, pgs 62-68). 

One of the reasons that AACSB-International (2006) is open to the use of a variety of methods in relation to 
assurance of learning is that no single “correct” approach exists. For example, a variety of course embedded 
measurement approaches are endorsed (e.g., papers and applied projects required to successfully compete a course) 
as are standardized achievement tests. The field of study concerned with examining and documenting the advantages 
and disadvantages associated with various assessment approaches is generically referred to as educational outcomes 
assessment (DeMong, Lindgren, & Perry, 1994; Palomba & Banta, 1999). In line with the recommendations of 
AACSB-International (2006), a wide range of methods have been proposed and used by universities to provide 
evidence of learning achievement (DeMong et al., 1994; Herring & Izard, 1992; Herring, Scheiner, & Williams, 
1989; Palomba & Banta, 1999; Paskow & Francis, 1990). 
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One of the standardized approaches to assessing learning achievement that has been successfully used is the 
Educational Testing Service (ETS) Major Field Achievement Test in Business (MFAT-B). This 120 item multiple 
choice assessment has been shown to correlate strongly with undergraduate student grade point average (GPA) in 
business core (GPA-BUS) classes (Allen & Bycio, 1997; Bycio & Allen, 2007). As such, performance on the 
MFAT-B provides a basis for comparing student achievement across business schools. 

Many, if not all business programs, have academic goals that go beyond providing business knowledge per 
se. For example, most academic programs aim to foster critical thinking. While many perspectives on the meaning 
of critical thinking exist, an international multidisciplinary panel characterized it as “the process of purposeful, self 
regulatory judgment. This process gives reasoned consideration to evidence, content, conceptualizations, methods, 
and criteria” (The APA Delphi Report, 1990). As the definition implies, critical thinking goes beyond the 
acquisition of knowledge and it is seen as a capability that has the potential to make an on-going contribution to 
quality of life in general (Facione, 2007). The California Critical Thinking Skills Test (CCTST; Facione, 2000), a 
34-item standardized exam was developed to assess critical thinking in line with the conceptualization forwarded by 
the APA Delphi Report (1990). Analysis, interpretation, and evaluation are among the attributes of interest 
identified by the Delphi group. 

THE CURRENT INVESTIGATION 

As part of an AACSB reaffirmation effort, the CCTST was administered at the authors’ business school. 
The test was used because critical thinking per se was one of six competencies that our faculty intended to develop in 
our undergraduate population. While the performance of the students was obviously of interest, there was a need to 
further examine the validity of the CCTST from an outcomes assessment perspective. In particular, those 
responsible for designing and implementing accreditation efforts need to examine the degree to which the CCTST 
provides useful assessment information beyond that provided by indices of academic aptitude such as the Scholastic 
Aptitude Test (SAT), or by business specific tests such as the MFAT-B. There are both theoretical and practical 
considerations involved. For example, the CCTST test requires 45 minutes of administration time and costs of 
approximately five dollars per examinee. These could be viewed as substantial depending upon the scope of the 
accreditation effort and the financial resources of a given school. As such, we examined the degree to which the 
most recent version of the CCTST (Form 2000) provided useful student outcomes information in the context of a 
broader AACSB reaffirmation effort. 

Very little published research pertaining to the CCTST exists beyond that provided in the 2002 CCTST test 
manual (Facione, Facione, Blohm, & Giancario, 2002). For example, we are unaware of any study that examines the 
test as part of a business school outcomes program. As such, this study is exploratory in nature. Below, we raise a 
series of questions involving (for example) the relationship of the CCTST to the SAT, student GPAs, and the 
MFAT-B. Data pertaining to these issues will be of interest to those designing AACSB outcomes efforts and to 
those looking to use the CCTST as a reflection of critical thinking per se. 

1. Not surprisingly, earlier forms of the CCTST (Forms A and B) were significantly related to academic 
aptitude as reflected by performance on SAT Verbal (SAT-V) and Math (SAT-M) exams. Since Form 
2000 retains 22 of the 36 items from the earlier versions, we expected similarly moderate associations 
between the SAT and the CCTST 2000. Nonetheless, if the associations are too strong, the CCTST might 
simply reflect a general capability to deal with standardized multiple choice academic questions (test taking 
ability, cf. DeMong et ah, 1994) as opposed to the intended broad cross-situational attribute described in 
the APA Delphi Report (1990). 

2. Moderate associations are expected between the CCTST and the MFAT-B because both are standardized 
multiple choice tests and because some of the MFAT-B items involve more than mere knowledge of 
business concepts given that application and interpretation are required as well. Nonetheless, as with the 
SAT, if the associations are especially strong, the incremental value of administering the CCTST as part of 
an AACSB accreditation effort would be called into question. 
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3. To the extent that CCTST performance reflects something more than test taking ability, we would expect it 
to relate to certain expects of a curriculum more strongly than others. For example, our undergraduate 
business curriculum ends with a capstone course that is intended (among other things) to assess student 
capabilities to analyze and respond to multifaceted business cases. Stated differently, the capstone course 
associated with each of our business majors is intended to place demands on students that better reflect the 
specific attributes that the CCTST is intended to measure (e.g., analysis, interpretation and evaluation). As 
such, we expect the CCTST and business capstone grade (CAPSTONE) relationship to be among the 
strongest observed in the study. 

4. Our business curriculum is nested within a substantial liberal arts program. Only half of the credits required 
for our undergraduate business degree are from the business college. Although some believe this has put 
our business school at a competitive disadvantage, the relative emphasis on liberal arts has not changed 
because of a belief that much of our non-business curriculum more directly fosters the development of many 
of the attributes targeted by the CCTST. Given the nature of this liberal arts and business mix, critical 
thinking as conceptualized on the CCTST would be expected to relate more strongly to performance in 
broad curriculum (cf., Facione, 2007) than to the business content only. As such, we expected the 
correlation between the CCTST and overall university GPA (GPA-U) to be significantly larger than the 
CCTST- GPA-BUS relationship. 

5. The CCTST 2000 manual (Facione et al., 2002) states that there is no reason to believe that the test unfairly 
discriminates among students based on their major. Nonetheless, some differences among business majors 
have been observed on other standardized tests. For example, with regard to the MFAT-B, Allen and Bycio 
(1997) reported that those in Accounting and Finance performed significantly better than students in other 
business majors. Importantly, these differences were mirrored by similar variations in SAT profiles across 
majors. Thus, it was unclear whether the differences by business major reflected test content biases or test 
taking ability. In any case, the Allen and Bycio (1997) finding leaves open the possibility of CCTST 
differences, especially if there are SAT variations across majors. 

6. Gender-based differences were not expected on the CCTST 2000 given that these were not an issue with the 
earlier Forms A and B (cf. Facione et al., 2002). Nonetheless, further investigation would be called for 
(e.g.. Buckless, Fipe, & Ravenscroft, 1991) if gender variations were observed. 

7. Student motivation has been found to predict performance on standardized tests in AACBC accreditation 
contexts. For example, Bycio and Allen (2007) asked students about the degree to which they were 
motivated to take the MFAT-B and the extent to which the maintenance of AACSB accreditation was 
important to them. They found that student motivation was associated with MFAT-B performance even 
after SAT and GPA variables were controlled for. Since the CCTST was part of the accreditation effort, we 
expected that the level of importance students ascribed to reaffirmation would be predictive in this context 
as well. 

METHOD 

Sample and Procedure 

All graduating business majors were required to complete a series of assessments as part of a AACSB 
reaffirmation effort. The first of these was the MFAT-B which was administered to the students in one large group 
(cf. Bycio and Allen, 2007). Prior to MFAT-B administration, the senior author delivered a 10 minute presentation 
to students concerning the importance of AACSB reaffirmation. Among other things, the long-term multifaceted 
nature of the accreditation process was explained and the AACSB website with the school accreditation status was 
displayed. The role that AACSB plays in U.S. News and World Report rankings of business programs was 
addressed as well. After the presentation, students were asked to rate the degree to which AACSB reaffirmation was 
important to them (see AACSB Importance below). 


3 



American Journal of Business Education - November 2009 


Volume 2, Number 8 


In subsequent weeks, the students were also required to complete the CCTST 2000. However, unlike the 
MFAT-B, the CCTST was administered to students in their respective capstone classes. To promote standard test 
conditions the senior author (a department chair at the time) visited each capstone class to oversee the process. 

Of the 185 graduating business majors, 162 (88%) were in attendance to take the CCTST. Consistent with 
university guidelines, students had the option to have their data excluded from this study. Thirty-one students (19%) 
indicated they did not want to be included. The resulting sample size was 131. Fifty-four percent were male and 
forty-six percent were female. 

Data Collection of Standardized Indices 

SAT scores and all GPAs were obtained directly from student records. The MFAT-B and CCSTS 2000 
results were collected as part of the AACSB reaffirmation effort. 

AACSB Importance 

Following a presentation concerning the role that AACSB plays in accrediting business schools, students 
were asked “As a graduating student, how important is it to you that the college of business continues to be 
accredited by the AACSB?” A four-point response scale was used (1 = “no importance” - 4 = “all important”). 

Analysis 


Testing five of the seven research questions required computing correlations and/or comparing the size of 
the correlations involving the CCTST and the various predictors. When the size of correlations were compared, the 
t-test formula for dependent correlations given by Cohen and Cohen (1983, pgs. 56-57) was used, for example, to 
determine if the CAPSTONE had, relatively speaking, a significantly stronger association with CCTST scores. 
Analysis of variance (cf. Winer, 1971) was used to test the two remaining questions. In particular, F-tests were used 
to evaluate the possibility of significant differences in CCTST by business major and gender. 

Also, multiple regression analysis (cf. Cohen & Cohen, 1983) was used to explore the degree to which the 
combining different types of variables (e.g., SAT, MFAT-B, GPAs, and AACSB-Importance) could account for 
CCTST performance. As a follow up, stepwise regression was employed to identify the smallest set of variables that 
could be used to optimally predict the CCTST. As such, it provides information concerning the degree to which 
each variable had a significant, unique association with CCTST scores (cf. Cohen & Cohen, 1983, pgs. 123-125). 

RESULTS 

Descriptive Statistics 

Table 1 shows the means, standard deviations and correlations among all the variables. The sample sizes 
vary due to missing data. This is especially relevant to findings involving the SAT, where n = 76. This difference 
relative to the GPA-based findings (where N = 131) reflects the fact that the SAT was not necessarily required to 
gain admission to the university. For clarity, in reporting the results below, the sample size associated with each 
individual finding is noted. 

Overall CCTST Performance 

The mean CCTST score in our sample was M = 20.33 (N = 112). This score was virtually identical to the 
mean of 20 reported by the testing company, which includes students who did not want their data used in this study. 
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Table 1: Descriptive Statistics and Correlations Among All Study Variables 




N 

M 

SD 

1 

2 

3 

4 

5 

6 

7 

1 . 

CCTST 

112 

20.33 

4.75 








27 

MFAT-B 

131 

161.78 

13.21 

.63 

(112) 







37 

SAT-V 

76 

562.24 

79.78 

.66 

(68) 

.66 
( 76) 






47 

SAT-M 

76 

572.24 

76.41 

.69 

(68) 

.64 

(76) 

.62 

(76) 





57 

GPA-BUS 

131 

3.21 

.56 

.47 

(112) 

. 62 
(131) 

.52 

(76) 

.59 

(76) 




67 

GPA-U 

131 

3.21 

.49 

.46 

(112) 

.59 

(131) 

.50 

(76) 

.59 

(76) 

.92 

(131) 



77 

CAPSTONE 

115 

3.24 

.68 

.21 

(97) 

.30 

(115) 

.36 

(64) 

.32 

(64) 

.43 

(115) 

.48 

(115) 


87 

AACSB-IMP 

131 

3.01 

.89 

.25 

(112) 

.25 

(131) 

.10 

(76) 

.21 

(76) 

.20 

(131) 

.15 

(131) 

.07 

(131) 


Note. The sample size associated with each correlation is presented in parentheses. When n = 76, correlations of .27 are 
significant at the p <.01 level. When N = 131, correlations of .23 are significant at thep < .01 level. 


SAT, MFAT-B, GPA, and CCTST Performance 

As Table 1 shows, both SAT-V (r = .66, p < .01, n = 68) and SAT-M (r = .69, p < .01, n = 68) are 
significant and moderately large predictors of CCTST performance. Moreover, CCTST was also strongly related to 
the MFAT-B (r = .63, p < .01, n = 112). However, contrary to expectations, the CCTST was not an especially 
strong predictor of performance in the business capstone course (r = .21, p < .05, n = 97). In fact, tests comparing 
dependent correlations revealed that both SAT-V (T(53) = -3.13, p < .05) and SAT-M (T(53) = -3.40, p < .05) 
were significantly better predictors of the CCTST than was the CAPSTONE. 

Finally, the CCTST scores predicted overall GPA-U (r = .41, p < .01, n = 97), but the strength of the 
relationship was not significantly better than that for GPA-BUS (r = .42 , p < .01, n = 97). 

CCTST Performance by Major and Gender 

Regarding the possibility of differences in CCTST performance across business majors, the authors 
confined their analysis to those specialties with at least 20 students (i.e.. Finance n = 27, Management- 
Entrepreneurship-Human Resources n = 23, and Marketing n = 34). There were no significant differences (F(2, 81) 
= 2.18, n.s.). Finally, analysis of variance pertaining to gender and the CCTST revealed no significant differences 
(F(l, 110) = .25, n.s.). 

Accreditation Importance and CCTST Performance 

As shown in Table 1, the mean student response regarding the importance of AACSB reaffirmation was 
3.01 out of 4. This is not especially high given the efforts of the senior author to explain the value of AACSB 
reaffirmation to the students. Nonetheless, the importance of accreditation was a significant predictor of CCTST 
performance (r = .25, p < .01, n = 112), although the size of the correlation was significantly smaller than most of the 
other predictors (see Table 1). 

SAT, MFAT-B, GPA, and Accreditation Importance as a Predictor Set 

As presented in Table 2, multiple regression was used to examine the degree to which our variable set made 
significant contributions to the prediction of CCTST performance. As a group, the predictors accounted for 60% of 
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Table 3 shows the stepwise regression findings. Only the two SAT-based predictors (SAT-M and SAT-V) 
were required to account for 56% of the CCTST variance. None of the GPAs, MFAT-B performance, or 
accreditation importance added significantly to the prediction. 


Table 2: SAT, GPA, and Motivation Variables As Predictors of CCTST Performance 



F 

P 

1. SAT-V 

65.60 

.01 

2. SAT-M 

19.32 

.01 

3. GPA-BUS 

.18 

n.s. 

4. GPA-U 

.03 

n.s. 

5. MFAT-B 

3.45 

n.s. 

6. AACSB-IMPT. 

1.35 

n.s. 

Total Model r 2 = 

.60 


Note. These results pertain to the sample with missing cases 

deleted n = 68. 


Table 3: Stepwise Multiple Regression Predicting CCTST Performance 

2 2 


Partial r Model r 

F 

P 

1. SAT-M 

.48 .48 

61.63 

.00 

2. SAT-V 

.08 .56 

11.86 

.00 


Note. These results pertain to the sample with missing cases deleted, n = 68 


DISCUSSION 

CCTST Performance and General Intellectual Ability 

As expected, significant associations were observed between CCTST 2000 scores and both the SAT-V and 
SAT-M. Our correlations (.66 and .68, respectively) were slightly higher than those reported for nursing students 
(.55 and .44 respectively) using an earlier form of the test (CCTST 2002 Manual; Facione et al., 2002 p. 20). Also 
as expected, the CCTST 2000 was significantly related to MFAT-B performance and to various GPAs (GPA-BUS 
and GPA-U). On the face of it, all of these findings are supportive for a test intended to assess critical thinking, as 
are the lack of major or gender differences in the scores. 

Notwithstanding the above positives, some important expectations were not supported. For example, the 
magnitude of the CCTST-CAPSTONE relationship was not significantly larger than the relationships involving 
overall GPAs. In fact, the SAT-CCTST associations noted earlier were significantly stronger than the CCTST- 
CAPSTONE (r = .21). Further, the CCTST had relationships that were equal in magnitude with GPA-U and GPA- 
BUS, even though approximately one half the former consisted of liberal arts courses. These findings are less 
positive for the CCTST because one would want a measure of critical thinking to have its strongest relationships with 
courses and curriculum intended to especially challenge students to move beyond “just the facts”. Instead, the 
equally strong relationships involving the SAT, the GPAs and the MFAT-B suggest that the CCTST is potentially 
strongly influenced by general test taking ability. This possibility was reinforced by the stepwise regression analysis 
where the SAT scores were the only predictors retained and they accounted for almost as much variance in the 
CCTST (56%) as the entire predictor set (60%). By comparison, Bycio and Allen (2007) found that SAT, GPA, and 
Student Motivation components each contributed significant unique variance to the prediction of the MFAT-B. 
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A limitation of this investigation is that the findings pertain only to one group of graduates at a single 
university. As far as we know, this is the only examination of the CCTST involving business students. 

Beyond generalizability issues per se, the design of our study was purely correlational which lacks the 
advantages associated with controlled experiments (cf. Cook & Campbell, 1979). As such, for example, some of our 
expectations may have not received support because the business capstone and liberal arts curricula did not place 
especially strong demands on critical thinking as assessed by the CCTST. Consistent with this possibility, in an 
experiment cited in the CCTST 2002 Test Manual (Facione et al. 2002; p. 19), undergraduates who took a semester- 
long course in critical thinking per se, performed significantly better on an earlier form of the CCTST than those who 
did not. Perhaps if our business faculty had been asked specifically to design content around the dimensions of 
critical thinking that underlie the CCTST, a stronger association to the capstone grades would have been observed. 
Future research concerning the CCTST should involve building faculty awareness and consensus around the 
conceptualization of critical thinking as put forward in the APA Delphi Report (1990). Finally, should greater 
confidence be established in the CCTST as a measure of critical thinking per se, it would be of obvious interest to 
employ a pre-post test experimental design (cf. Cook & Campbell, 1979) to evaluate the degree of change associated 
with the completion of a business degree. 

CONCLUSION 

Confirmation of moderately strong relationships between general intellect and CCTST performance is 
important because it implies that schools with strong SAT profiles will likely have a performance edge relative to 
those with less stringent admission requirements. As such, from a macro perspective, CCTST scores cannot be 
meaningfully compared across institutions without also accounting for possible differences in SAT indices. 
Similarly, since AACSB accreditation importance was significantly related to CCTST performance, differences in 
student motivation must also be accounted for when making comparisons across business schools. Importantly, these 
cautions are not unique to the CCTST. Similar issues are associated with the interpretation of the MFAT-B as well 
(cf., Bycio & Allen (2007). 
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